Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[tools] Add opt-in build flags (and unit test) for OpenMP #16606

Merged
merged 1 commit into from
May 5, 2022

Conversation

jwnimmer-tri
Copy link
Collaborator

@jwnimmer-tri jwnimmer-tri commented Feb 16, 2022

Towards #14858.


This change is Reviewable

@jwnimmer-tri
Copy link
Collaborator Author

jwnimmer-tri commented Feb 16, 2022

Interesting, lots of timeouts. Perhaps one of our third-party dependencies has OpenMP parallelization directives lying dormant.

It looks like cmaes, csdp, and scs all mention OpenMP so will need some investigation.

For cmaes, the omp-parallel-for is opt-in per a configuration flag, so is not a problem here.

For csdp and scs, it looks like omp is always armed, so we'll need an RAII guard before calling it in cases where we need to keep it single-core. Or actually, none of their omp is in header files, so I think we could also do copts = ["-fno-openmp"] in their build file to opt-out.

@jwnimmer-tri jwnimmer-tri force-pushed the omp branch 2 times, most recently from 4df4f29 to 57deeb6 Compare February 17, 2022 19:02
@jwnimmer-tri jwnimmer-tri force-pushed the omp branch 2 times, most recently from 710c47e to 1ba26e1 Compare February 22, 2022 23:21
@jwnimmer-tri jwnimmer-tri changed the title Spike test for OpenMP [tools] Add opt-in build flags (and unit test) for OpenMP Apr 28, 2022
@jwnimmer-tri jwnimmer-tri force-pushed the omp branch 2 times, most recently from 820a2b8 to 11a4490 Compare April 28, 2022 16:37
@jwnimmer-tri jwnimmer-tri added priority: medium release notes: none This pull request should not be mentioned in the release notes and removed status: do not merge status: do not review labels Apr 28, 2022
@jwnimmer-tri jwnimmer-tri marked this pull request as ready for review April 28, 2022 16:37
@jwnimmer-tri
Copy link
Collaborator Author

+@calderpg-tri for feature review of the C++ code please (the new unit test).
+@rpoyner-tri for feature review of the everything else, please (the build system & doc changes).

(Either of you should feel free to comment on anything in the pull request, but please cover those areas specifically.)

Copy link
Contributor

@calderpg-tri calderpg-tri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 2 of 7 files at r1, 9 of 9 files at r2, all commit messages.
Reviewable status: 2 unresolved discussions, LGTM missing from assignees rpoyner-tri(platform),calderpg-tri (waiting on @jwnimmer-tri and @rpoyner-tri)


common/test/openmp_test.cc line 20 at r2 (raw file):

#else
constexpr bool kHasOpenmp = false;
// TODO(jwnimmer-tri) This should probably be a Drake helper function wrapper

fyi you can use or wrap equivalent helpers from common_robotics_utilities/openmp_helpers.hpp


common/test/openmp_test.cc line 34 at r2 (raw file):

  // Populate the storage, in parallel.
  constexpr int num_threads = 4;

Is there a reason for manually specifying the number of threads here and the number of cpus for the test, versus specifying OMP_NUM_THREADS=4 for the test and letting the omp parallel use all available threads? Or, alternatively something like CRU'sChangeOmpNumThreadsWrapper(4)/an explicit call to omp_set_num_threads(4)?


common/test/openmp_test.cc line 42 at r2 (raw file):

  }

  // Confirm how many threads were used.

btw This seems to be testing both that the expected number of threads were used, and that their thread numbers were the expected [0] or [0, 1, 2, 3]. If that is intended, perhaps the comment should reflect this. (If we only cared about number of threads run, presumably something more direct like

EXPECT_EQ(std::unordered_set<int>(outputs.begin(), outputs.end()).size(), kHasOpenmp ? num_threads : 1)

would be enough)


tools/workspace/csdp/package.BUILD.bazel line 57 at r2 (raw file):

    ],
    copts = [
        "-fno-openmp",

It looks like OpenMP was already disabled in these copts due to the use of longjmp

@jwnimmer-tri
Copy link
Collaborator Author

a discussion (no related file):
Working

The CI error /usr/bin/ld.gold: error: cannot find -lomp is because the Clang 12 transition is still in flight. (We can only have one of Ubuntu Clang's OMP development packages installed concurrently, and for the moment that's omp-12.)


Copy link
Collaborator Author

@jwnimmer-tri jwnimmer-tri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 3 unresolved discussions, LGTM missing from assignees rpoyner-tri(platform),calderpg-tri (waiting on @calderpg-tri, @jwnimmer-tri, and @rpoyner-tri)


common/test/openmp_test.cc line 20 at r2 (raw file):

Previously, calderpg-tri wrote…

fyi you can use or wrap equivalent helpers from common_robotics_utilities/openmp_helpers.hpp

Yup, I had that in mind but should have written it down, too.


common/test/openmp_test.cc line 34 at r2 (raw file):

Previously, calderpg-tri wrote…

Is there a reason for manually specifying the number of threads here and the number of cpus for the test, versus specifying OMP_NUM_THREADS=4 for the test and letting the omp parallel use all available threads? Or, alternatively something like CRU'sChangeOmpNumThreadsWrapper(4)/an explicit call to omp_set_num_threads(4)?

That would probably be an improvement. I'll take it for a spin; stay tuned.


common/test/openmp_test.cc line 42 at r2 (raw file):

Previously, calderpg-tri wrote…

btw This seems to be testing both that the expected number of threads were used, and that their thread numbers were the expected [0] or [0, 1, 2, 3]. If that is intended, perhaps the comment should reflect this. (If we only cared about number of threads run, presumably something more direct like

EXPECT_EQ(std::unordered_set<int>(outputs.begin(), outputs.end()).size(), kHasOpenmp ? num_threads : 1)

would be enough)

Done.


tools/workspace/csdp/package.BUILD.bazel line 57 at r2 (raw file):

Previously, calderpg-tri wrote…

It looks like OpenMP was already disabled in these copts due to the use of longjmp

Done.

Copy link
Contributor

@calderpg-tri calderpg-tri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 2 of 2 files at r3, all commit messages.
Reviewable status: 2 unresolved discussions, LGTM missing from assignees rpoyner-tri(platform),calderpg-tri (waiting on @jwnimmer-tri and @rpoyner-tri)

Copy link
Contributor

@rpoyner-tri rpoyner-tri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed all commit messages.
Reviewable status: 2 unresolved discussions, LGTM missing from assignees rpoyner-tri(platform),calderpg-tri (waiting on @jwnimmer-tri)

Copy link
Contributor

@rpoyner-tri rpoyner-tri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 2 of 7 files at r1, 3 of 9 files at r2, 1 of 2 files at r3.
Reviewable status: 2 unresolved discussions, LGTM missing from assignee calderpg-tri (waiting on @jwnimmer-tri)

Copy link
Contributor

@rpoyner-tri rpoyner-tri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 2 of 2 files at r4, all commit messages.
Reviewable status: 2 unresolved discussions, LGTM missing from assignee calderpg-tri (waiting on @jwnimmer-tri)

Copy link
Collaborator Author

@jwnimmer-tri jwnimmer-tri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 2 unresolved discussions, LGTM missing from assignee calderpg-tri (waiting on @calderpg-tri and @jwnimmer-tri)


common/test/openmp_test.cc line 34 at r2 (raw file):

Previously, jwnimmer-tri (Jeremy Nimmer) wrote…

That would probably be an improvement. I'll take it for a spin; stay tuned.

Done.

I also reduced this test to just 2 threads, in case anyone ever runs it on a potato that doesn't have 4 cpus.

Copy link
Contributor

@rpoyner-tri rpoyner-tri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 4 of 9 files at r2, 1 of 2 files at r3.
Reviewable status: 2 unresolved discussions, LGTM missing from assignee calderpg-tri (waiting on @calderpg-tri and @jwnimmer-tri)

Copy link
Contributor

@calderpg-tri calderpg-tri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 2 of 2 files at r4, all commit messages.
Reviewable status: 1 unresolved discussion (waiting on @jwnimmer-tri)

@jwnimmer-tri jwnimmer-tri mentioned this pull request May 2, 2022
@jwnimmer-tri
Copy link
Collaborator Author

+@xuchenhan-tri for platform review per schedule, please.

I expect the CI failure to resolve on its own with some infrastructure changes, no more code changes planned for this PR.

Copy link
Contributor

@xuchenhan-tri xuchenhan-tri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

platform :lgtm:

Reviewed 2 of 7 files at r1, 2 of 9 files at r2, 2 of 2 files at r4, all commit messages.
Reviewable status: 1 unresolved discussion (waiting on @jwnimmer-tri)

@jwnimmer-tri
Copy link
Collaborator Author

@drake-jenkins-bot linux-focal-clang-bazel-experimental-everything-release please

At the moment '--config omp' is not covered by CI.
Soon it will become part of the "Everything" builds.
Copy link
Collaborator Author

@jwnimmer-tri jwnimmer-tri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! all discussions resolved, LGTM from assignees rpoyner-tri(platform),calderpg-tri,xuchenhan-tri(platform) (waiting on @calderpg-tri, @rpoyner-tri, and @xuchenhan-tri)

a discussion (no related file):

Previously, jwnimmer-tri (Jeremy Nimmer) wrote…

Working

The CI error /usr/bin/ld.gold: error: cannot find -lomp is because the Clang 12 transition is still in flight. (We can only have one of Ubuntu Clang's OMP development packages installed concurrently, and for the moment that's omp-12.)

Given the current state of Clang in CI, there is no good way to deal with it in this PR. I'm going to add a GCC-specific hack to drake-ci shortly, so that we can at least have CI coverage of that starting point.


@jwnimmer-tri jwnimmer-tri merged commit 4c5990a into RobotLocomotion:master May 5, 2022
@jwnimmer-tri jwnimmer-tri deleted the omp branch May 5, 2022 17:52
aykut-tri pushed a commit to aykut-tri/drake that referenced this pull request May 27, 2022
…otion#16606)

At the moment '--config omp' is not covered by CI.
Soon it will become part of the "Everything" builds.
aykut-tri pushed a commit to aykut-tri/drake that referenced this pull request Jun 1, 2022
…otion#16606)

At the moment '--config omp' is not covered by CI.
Soon it will become part of the "Everything" builds.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority: medium release notes: none This pull request should not be mentioned in the release notes
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

4 participants